Multiclass Boosting with Hinge Loss based on Output Coding

Multiclass classification is an important and fundamental problem in machine learning. A popular family of multiclass classification methods belongs to reducing multiclass to binary based on output coding. Several multiclass boosting algorithms have been proposed to learn the coding matrix and the associated binary classifiers in a problemdependent way. These algorithms can be unified under a sum-of-exponential loss function defined in the domain of margins (Sun et al., 2005). Instead, multiclass SVM uses another type of loss function based on hinge loss. In this paper, we present a new outputcoding-based multiclass boosting algorithm using the multiclass hinge loss, which we call HingeBoost.OC. HingeBoost.OC is tested on various real world datasets and shows better performance than the existing multiclass boosting algorithm AdaBoost.ERP, one-vsone, one-vs-all, ECOC and multiclass SVM in a majority of different cases.